Effective Language Representations for Danmaku Comment Classification in Nicovideo

Danmaku commenting has become popular for co-viewing on video-sharing platforms, such as Nicovideo. However, many irrelevant comments usually contaminate the quality of information provided by videos. Such an pollutant problem can be solved a comment classifier trained with abstention option, which detects whose video categories are unclear. To improve performance this classification task, paper presents Nicovideo-specific language representations. Specifically, we used sentences from Nicopedia, Japanese online encyclopedia entities that possibly appear in Nicovideo contents, to pre-train bidirectional encoder representations Transformers (BERT) model. The resulting model named Nicopedia BERT is then fine-tuned it could determine whether given falls into any predefined categories. experiments conducted data demonstrated effectiveness compared existing models pre-trained using Wikipedia or tweets. We also evaluated each additional sentiment and obtained results implied applicability feature extractor other social media text.